New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

perf: optimize circom proof generation #469

Merged

chokobole merged 21 commits into main from perf/optimize-circom-proof-generation

Jul 15, 2024

Contributor

chokobole commented Jul 14, 2024

Description

This PR optimizes circom proof generation by avoiding parsing zkey and wtns files.

Related: #460

chokobole requested review from batzor, dongchangYoo, fakedev9999, GideokKim and ashjeong

July 14, 2024 15:40

chokobole force-pushed the perf/optimize-circom-proof-generation branch from ccd330e to 7531107 Compare

July 14, 2024 15:44

chokobole marked this pull request as ready for review

July 14, 2024 15:45

chokobole force-pushed the perf/optimize-circom-proof-generation branch from 7531107 to bd59258 Compare

July 14, 2024 16:00

fakedev9999 commented Jul 15, 2024

In 134c146, "not moved" -> "not be moved" would be better.

fakedev9999 commented Jul 15, 2024

In 63d388a, there is a typo "pref" -> "perf".

fakedev9999 reviewed

View reviewed changes

tachyon/base/maybe_owned.h Show resolved Hide resolved

tachyon/base/buffer/read_only_buffer.h Show resolved Hide resolved

chokobole force-pushed the perf/optimize-circom-proof-generation branch 3 times, most recently from 9ad620b to 84b4bd3 Compare

July 15, 2024 03:30

batzor reviewed

View reviewed changes

tachyon/zk/r1cs/groth16/verifying_key.h Outdated Show resolved Hide resolved

batzor reviewed

View reviewed changes

tachyon/zk/r1cs/groth16/prepared_verifying_key.h Outdated Show resolved Hide resolved

batzor reviewed

View reviewed changes

tachyon/zk/r1cs/groth16/proving_key.h Outdated Show resolved Hide resolved

batzor reviewed

View reviewed changes

tachyon/zk/r1cs/groth16/proving_key.h Outdated Show resolved Hide resolved

batzor reviewed

View reviewed changes

vendors/circom/circomlib/wtns/wtns_unittest.cc Outdated Show resolved Hide resolved

chokobole force-pushed the perf/optimize-circom-proof-generation branch from 84b4bd3 to 46acc2f Compare

July 15, 2024 04:08

chokobole added 6 commits

July 15, 2024 13:11


          feat(base): add MaybeOwned

4ca776d


          feat(zk): add To(Hex)String() to VerifyingKey

9ea9457


          feat(zk): add To(Hex)String() to PreparedVerifyingKey

cbcdf85


          feat(zk): add To(Hex)String() to ProvingKey

93e2b91


          feat(math): add To(Hex)String() to G2PreparedBase

2774b55


          feat(zk): change VerifyingKey to hold MaybeOwned<T>

b93b47b

This adjustment aims to remove the overhead associated
with parsing zkey.

chokobole force-pushed the perf/optimize-circom-proof-generation branch from 46acc2f to 589c16b Compare

July 15, 2024 04:11

fakedev9999 reviewed

View reviewed changes

vendors/circom/README.md Show resolved Hide resolved

chokobole force-pushed the perf/optimize-circom-proof-generation branch from 589c16b to aab44e1 Compare

July 15, 2024 04:44

batzor approved these changes

View reviewed changes

Contributor

batzor left a comment

LGTM

GideokKim reviewed

View reviewed changes

tachyon/zk/r1cs/groth16/proving_key.h Show resolved Hide resolved

vendors/circom/circomlib/zkey/zkey.h Show resolved Hide resolved

benchmark/msm/README.md Outdated Show resolved Hide resolved

fakedev9999 approved these changes

View reviewed changes

fakedev9999 left a comment

LGTM

chokobole added 7 commits

July 15, 2024 15:12


          feat(zk): change ProvingKey to hold MaybeOwned<T>

b2b8fa7

This adjustment aims to remove the overhead associated
with parsing zkey.


          feat(math)!: remove alignment from BigInt

f51f57f

BREAKING CHANGE: The alignment requirement for `BigInt` is removed.
This alignment was originally inherited from the era-bellman-cuda project
(see: https://github.com/matter-labs/era-bellman-cuda/blob/b52127574e67f4c7938890eab0670ae75f49c22f/src/ff_storage.cuh#L6).
The specific reason for this alignment requirement is unclear,
and its presence necessitates parsing the ZKey.


          fix: avoid moving VerifyingKey in TachyonRunner

26a8c35

Since `VerifyingKey` is used in proof generation, it should not be moved.


          perf(zk): use std::move() where possible

c759475


          feat(base): add ReadPtr(At)() to ReadOnlyBuffer

375d63e


          perf(circom): avoid parsing ProvingKey from zkey file

929f666


          fix(circom): use OMP_FOR instead of OMP_FOR_NOWAIT

a53163a

To ensure that `c[i] = a[i] * b[i]` is executed correctly, `a[i]` and `b[i]`
must be fully computed.

dongchangYoo approved these changes

View reviewed changes

Contributor

dongchangYoo left a comment

LGTM

chokobole force-pushed the perf/optimize-circom-proof-generation branch from aab44e1 to ac858d5 Compare

July 15, 2024 08:46

GideokKim approved these changes

View reviewed changes

Contributor

GideokKim left a comment

LGTM

ashjeong requested changes

View reviewed changes

vendors/circom/circomlib/zkey/coefficient.h Outdated Show resolved Hide resolved

vendors/circom/circomlib/zkey/coefficient.h Outdated Show resolved Hide resolved

vendors/circom/circomlib/zkey/coefficient.h Outdated Show resolved Hide resolved

chokobole added 8 commits

July 15, 2024 18:01


          perf(circom): optimize ConstraintMatrices parsing

722ff8c

1. Avoid parsing into intermediate `circom::ConstraintMatrices`.
   Instead, it just takes pointer to `circom::Coefficient`.
2. Parallelize dividing by R.
3. Avoid computing unused `a_num_non_zero` and `b_num_non_zero`.


          perf(circom): avoid parsing Wtns from wtns file

3b99a5c


          docs(benchmark): add missing command to run MSM with CUDA

a6a9b46


          feat(math): add IcicleNTTOptions

3da92a5


          feat(math): add IcicleMSMOptions

acb004b


          docs(circom): add missing flag description

5dfdd27


          feat: add disable_fast_twiddles_mode flag

a867473


          fix(zk): use num_witness_variables() instead of num_constraints()

6862c02

chokobole force-pushed the perf/optimize-circom-proof-generation branch from ac858d5 to 6862c02 Compare

July 15, 2024 09:01

ashjeong approved these changes

View reviewed changes

Contributor

ashjeong left a comment

LGTM

chokobole merged commit f9a40b0 into main

3 checks passed

chokobole deleted the perf/optimize-circom-proof-generation branch

July 15, 2024 09:56

dongchangYoo pushed a commit that referenced this pull request


          fix(rs): remove obsolete alignments

ea72a26

This is a missing change from PR #469.

dongchangYoo pushed a commit that referenced this pull request


          fix(rs): remove obsolete alignments

fa25ebe

This is a missing change from PR #469.

dongchangYoo pushed a commit that referenced this pull request


          fix(rs): remove obsolete alignments

07aa328

This is a missing change from PR #469.

chokobole added a commit that referenced this pull request


          fix(rs): remove obsolete alignments

768b4ba

This is a missing change from PR #469.

dongchangYoo pushed a commit that referenced this pull request


          fix(rs): remove obsolete alignments

07092ad

This is a missing change from PR #469.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

ashjeong ashjeong approved these changes

GideokKim GideokKim approved these changes

fakedev9999 fakedev9999 approved these changes

dongchangYoo dongchangYoo approved these changes

batzor batzor approved these changes

Labels

None yet